Data compression involves encoding information using fewer bits than the original representation. Data compression techniques may be utilized to reduce data storage and retrieval expenses, improve data query performance, and provide deeper analytics than what would be available using uncompressed data.
For example, cloud computing has emerged as a preferred avenue for storing data. However, the cost associated with storing data on the cloud is proportional to the amount of data being stored—the more data, the higher the cost. In addition, there is a cost associated with retrieving data from the cloud—the more frequently data is retrieved, the higher the cost. Thus, data storage and retrieval costs may prove prohibitive for many entities. This is especially true in the age of “big data,” where entities may need to store, quickly access, and quickly analyze massive amounts of data.
Data compression techniques may also be utilized to reduce disk I/O because less data blocks are used to save the data. This has the potential of improving query execution performance overall, because less time is spent waiting for disk I/O when answering a query.
Further still, data compression techniques may be leveraged to provide deeper answers to business questions in an application service provider environment by furnishing access to a larger set of historical data. Storing tenant data for multiple years (e.g., 10 years) instead of only a few years (e.g., 2 years) provides a larger and richer data powered ecosystem for data mining and other analytical endeavors. Such richer insights from data mining efforts may be incorporated into the data available for answering business questions.
Conventional data compression techniques have proven suboptimal for compressing data in the “big data” era. For example, many conventional data compression techniques rely on a static configuration of compression filters arranged in a static, pre-determined sequence. While such static configurations may perform suitably on some types of data, these configurations perform sub-optimally on other types of data. This is particularly true as it relates to conventional data compression techniques abilities to compress patterned numerical data. Accordingly, improved methods and computing systems for performing high-density compression on patterned numerical data are desired.